excludes: say when a prune entry no longer matches anything - #2313

One of two PRs adding drift detection between this repo and OpenIPC/builder, after #2308 (comment) turned out to have three separate instances behind it. This is the cheap half and it lives here because it benefits every consumer of rootfs_script.sh, not just builder. The other half is OpenIPC/builder#129.

The problem

scripts/excludes/<model>_<variant>.list names files by hand, so it goes stale in one direction with nothing to show for it: a package renames or drops a sensor blob, the entry that used to prune it now prunes nothing, and the board keeps paying for whatever replaced it. The step was one line:

xargs -a ${LIST} -I % rm -f ${TARGET_DIR}%

which cannot tell "removed a file" from "removed nothing" — and fed the # separator lines these lists use to rm as literal paths besides.

The case that prompted it

OpenIPC/builder's hi3518ev200_lite.list names 25 sensor .so files where hisilicon-osdrv-hi3516cv200 now ships 17. Eight entries have been pruning nothing for however long. Meanwhile six sensor libraries it does not name — 188KB uncompressed — ship on a camera with one sensor:

libsns_gc2033.so  36K   libsns_imx323_i2c_dc_v2.so  44K   libsns_imx327.so  20K
libsns_imx307.so  20K   libsns_imx323_i2c_dc_v3.so  44K   libsns_sc2232.so  24K

That device is 4KB over its rootfs cap. Nothing in any build log said so.

What changes

Each entry is checked before removal, misses are named, and a count is printed at the end:

excludes: /usr/lib/sensors/libsns_gone.so matched no file
excludes: 1 of 4 entries in testsoc_testvariant.list matched no file

Report, never fail. An image carrying a few kB it meant to drop is something to look at, not a reason to break a build. Boards whose lists are accurate print nothing new, so this is silent for everyone it does not concern.

The loop also fixes two things the xargs form got away with by accident rather than intent: comment and blank lines are skipped explicitly, and a final entry with no trailing newline is still read.

Deliberately not here: reporting files that ship but are not named. That needs the built image and a notion of which directories a list is responsible for, and it would fire on every board at once. This half is free.

Test

test_excludes_report.sh drives the real rootfs_script.sh against a synthetic TARGET_DIR — no build:

$ bash .github/scripts/test_excludes_report.sh
ok   exit status is 0
ok   present entry was removed
ok   entry on a line without a trailing newline was removed
ok   dangling symlink was removed
ok   absent entry is reported
ok   summary counts 1 stale of 4 real entries
ok   comment lines are not treated as paths
ok   excludes step still routes the list through a reporting loop

Against the pre-change script it fails 4 of 8 and passes the other 4:

$ SCRIPT_UNDER_TEST=/tmp/old_rootfs.sh bash .github/scripts/test_excludes_report.sh
ok   exit status is 0
ok   present entry was removed
ok   entry on a line without a trailing newline was removed
ok   dangling symlink was removed
FAIL absent entry was not reported
FAIL summary line wrong or missing: <none>
FAIL a comment line was treated as a path
FAIL excludes step no longer reports stale entries -- reverted to a bare xargs?

The four that pass are the part worth stating: removal behaviour is unchanged, including the dangling symlink (-e is false for one, so there is an explicit -L arm) and the unterminated last line. Only the reporting is new.

Part 2 of the test is a drift check anchored on the loop rather than the message text, so wording can change freely but a revert to a bare xargs fails immediately.

Test plan

• bash .github/scripts/test_excludes_report.sh — 8/8
• same test against the pre-change script — 4 fail, 4 pass, as above
• python3 .github/scripts/lint-workflow-shell.py — 49 run blocks parse clean
• python3 .github/scripts/ci-matrix.py --self-test — ok (99 boards, 132 packages, 52 cases)
• bash .github/scripts/test_sysupgrade.sh — all checks passed
• bash -n general/scripts/rootfs_script.sh
• Two clean builds of ssc333_lite_meari-speed-6s proving the pruning is unchanged — see below

Proof that pruning is unchanged

rootfs_script.sh runs on the build host and can only ever rm -f listed paths, so the claim to test is whether the set of removed files moved. Two clean builds of ssc333_lite_meari-speed-6s — a device that actually has an excludes list — from identical trees differing only in general/scripts/rootfs_script.sh:

Target file lists are identical: 665 paths, no diff. That is the only thing rm -f can influence.

Every remaining byte difference between the two trees is accounted for and none of it is this change:

9 of the 11 differing binaries are byte-identical once the build-id note is masked; the other two differ only in an embedded clock string. The script never rewrites a file, so none of this is reachable from it.

No board in this repository has an excludes list. general/scripts/excludes/ does not exist here, so [ -f "${LIST}" ] is false for all 99 boards and the changed branch never executes for any of them. The consumers are OpenIPC/builder's retail devices — which is why the verification above uses one.

And it earns its keep immediately on that device:

excludes: /etc/sensors/sc2338.bin matched no file
excludes: /lib/modules/4.9.84/sigmastar/sensor_jxq03_mipi.ko matched no file
excludes: /lib/modules/4.9.84/sigmastar/sensor_sc2338_mipi.ko matched no file
excludes: 3 of 36 entries in ssc333_lite.list matched no file

Three stale entries in a shipping device's list that nothing could have surfaced before.